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Executive summary 


The COVID-19 pandemic resulted in school closures around the world, at its peak affecting 1.6 billion students 
worldwide (UNESCO, 2020). This crisis has underscored the importance of resilient education systems that 
can support all, especially the most marginalized, to learn during times of crisis. Even before the COVID-19 
pandemic, nearly one in four children globally were living in countries affected by conflict or disaster, unable 
to routinely access quality education (UNICEF 2016). The growing use of digital technologies in education 
(commonly referred to as EdTech) during COVID-19 school closures has heighted the need to understand how 
to best use EdTech in humanitarian and crisis contexts. 


This report presents findings from the UNICEFAkelius Foundation Innovation in Education Partnership and its 
implementation of a digital course used on tablets and mobile phones for language learning in Lebanon, where 
40 per cent of school-age children and adolescents are Syrian refugees. The digital course was introduced in 
non-formal education (NFE) classes for Syrian refugees to strengthen English or French language learning and 
help their transition to Lebanon's trilingual education system, where classes are taught in Arabic, French and 
English. 


The goal of the digital course examined in this research is to provide students and teachers with a tool to 
accelerate language learning for marginalized children. The digital course is free, includes no advertising and 
requires no prior user information to access. It can be accessed online via a web browser, or online and 
offline (when content is downloaded) through a mobile application on tablets or mobile phones. The content 
and features of the course are developed through a co-creation approach with frequent communication and 
feedback from implementing teachers based on the real-world use of the course with students (see Image 1). 


Image 1. Examples of the Akelius Digital Language Course content 


500 words math 


When first implemented in Lebanon in 2019, the course was used in three centres serving 246 students as a 
teaching tool within traditional face-to-face classrooms. In class, students used the course on individual tablets 
with headphones and were guided by teachers who were present. However, when schools and education 
centres in Lebanon closed in March 2020, the course was implemented in a fully remote learning environment 
and expanded across the country to 64 NFE centres covering a total of 7237 refugee students. 


As a result, this report presents analysis on the implementation of the digital course in three key areas: 
First, the report investigates the digital course's use in a blended learning environment where it was used on 
tablets by students as part of traditional face-to-face classroom instruction with teachers. Second, the analysis 
examines the transition to remote learning where the course was used on devices owned by the household, 
supported by teachers remotely. Third, the report estimates the effectiveness of the use of the digital course 
during this period of remote learning from August-November 2020. 
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Summary of key findings: 


1. When used during in-person classes, qualitative evidence from teachers indicates that the course led 
to improvements in their classroom environment, students’ confidence, motivation to go to class and 
speaking skills. 


2. The successful transition to remote digital learning relied on frequent, clear communication between 
educators and households and ongoing support from educators to overcome challenges, including 
unreliable internet connectivity and distrust of technology within the community. 


3. In both face-to-face and remote settings, teachers remained the key to successful implementation 
of digital learning. When centres closed, teachers gave additional support to students and families 
beyond their already challenging role of teaching from a distance. They provided psychosocial support to 
families, as well as technological support by developing and delivering guides on how to use technology, 
providing data package top-ups and even helping to install internet routers within communities. 


4. When implemented in a fully remote learning environment, students from classes that introduced the 
digital course showed statistically significant improvements in foreign language (French or English, 
21.2 per cent increase; Arabic, 21.1 per cent increase), and art (11.1 per cent increase) competencies 
compared to students from centres that didn’t use the digital course. These findings are robust when 
controlling for baseline learning levels of students, demographic characteristics and the location of the 
learning centre. 
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atcxexe) aq} aatsaver-iulelalsmce) am ele) i(eavAnat-],<-1a-ym=lel 0 ler-)dle)am e)a-leadi tle) alc) a-yar-ale| 
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Education systems and practitioners were unprepared for the unprecedented school 

(ol(ols1U] gotomelUlo mo Mu alom @1@)V41 Dabo mol-lalel-laallom-lareme-) el(el\varelel-] elccre mace elce)iale(cmicy-laalialemielmenyileln-lar 
Governments need to build resilience in education systems and, as part of that process, 
have a systems approach to providing equitable digital learning whether the classroom is 
physical or virtual. The following are recommendations from the experience of the UNICEF 
Akelius Foundation partnership in Lebanon: 


= Governments should prioritize investments in reliable electricity and connectivity 
infrastructure, especially for hard-to-reach and marginalized areas and populations. 
Work must be done at the same time with the private sector to ensure low-cost and 
affordable data, all of which are needed to enable equitable digital learning. Education 
partners across Lebanon cited connectivity as the major barrier faced when providing 
folfelicslmaclaatolccm (cr-lanlialepmere)an] exelelate(-tom onand alc Malle] amelessimo)mer-)i-F 


Frequent communication with families and communities should continue beyond 
remote learning to sustain increased caregiver engagement in their children’s 
learning. Education coordinators noted that caregivers were more engaged in their 
children’s learning while using the digital course at home. For refugee families with low 
trust of institutions, the trust and relationships built with community-based NFE actors 
(oxo}alulalUlcyomicon eL-molgii(er-| MKem=)alecolele-lelomrclaalll(otsmcom olU/aclU[-m-lel0 lor 1t(o)ammielmcalclimeallieliclar 


Education providers must invest in the digital skills of teachers and facilitators and 
their readiness for digital learning, whether in the remote or physical classroom. 
Implementation of digital learning during school closures varied between centres, 
partners, and teachers. Many times those differences were based on the capacities of 
teachers and various technological constraints in communities. Investments in the digital 
skills of teachers and facilitators need to be met with ongoing support and guidance for 
them on how to effectively integrate technology into their teaching practice. 


The support system needed to enable digital learning must be invested in, 
and the key role of teachers in successful implementation of EdTech must be 
acknowledged and supported. Teachers provided extra-academic support to families 
in alleviating technology and connectivity issues throughout school closures. The use 
of technology for learning requires much more than a device held by a student and an 
engaged teacher. Governments and education providers should cost and invest in the entire 
S10] 0) ole) marsnZcikc laa mce)mel(elics) m(ct-1aaliare palace Ulell are mi amalelant-lamactsvelelcer-tomcome]i(o\al-icomecreralare) (ele iere]| 
and connectivity challenges, monitoring and evaluation systems, and implementation 
research to understand how digital learning works. Doing so would reduce the burden 
that falls on individual teachers or organizations to provide technology support on top of 
their regular teaching duties. 


Yo) a aUte-la-Me(-\VZ-1(0) oL-1e-MdiLelei(emel--1fe lam olaeleia-Tiat-me-larem olatelaiar4-mr-]oyeliter-hitelamict-Waele-t) 
that align with the needs of teachers and to the constraints of the learning 
environment. While interactive and engaging content should be a priority, it must be 
coupled with EdTech solutions designed to be used in low- or no-connectivity settings. 
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1. Introduction 


The use of EdTech in education has been long advocated for its potential to improve student outcomes through 
several pathways. Software can be designed to encourage and support self-paced learning that is typically 
difficult to achieve in group instruction, especially large classes (Koedinger et al., 1997; Roschelle et al., 2000; 
Rodriguez-Segura, 2021). Computer Assisted Learning (CAL) programs can enable adaptive learning where 
education content delivered on the screen is individualized based on the strengths and weaknesses of the 
student (Muralidharan et_al., 2019). While EdTech programmes may have a wide variety of goals, such as 
improving literacy and numeracy, or language skills, the use of EdTech can also help build students’ digital 
skills, which is increasingly important as economies transform and become more digital (Flack et al., 2020). 


However, while the potential is great for EdTech to support good practices in education and pedagogy, such 
as teaching at the right level, the evidence on effective use of technologies to improve children’s outcomes 
is mixed. What has become clear is that the introduction of technology has had large and positive effects for 
students and teachers when introduced into a clear teaching framework (Brossard et al., 2021; Bulman and 
Fairlie, 2016). In short, while the choice of the technological tool or software is important, how it is effectively 
used by teachers and students to achieve learning outcomes is even more critical. 


In the context of education in emergencies, the use of mobile technologies can also enable access to learning 
opportunities when education systems are disrupted (Iauson and Stannard, 2018; De Hoop et_al., 2019). 
In 2020, the COVID-19 pandemic and related school closures accelerated the take-up and use of mobile 
technologies to support children's learning. These education institutions’ closures have also highlighted the 
considerable inequities of children’s access to the technologies needed for remote learning, and the lack of 
preparedness of education systems across the globe to effectively use technology for learning (UNICEF, 2020; 
UNICEF 2021). So, while EdTech solutions have the potential to address education challenges in low-resource 
and conflict settings, where class sizes may remain large and learning levels remain low, (UNICEF 2019; Save 
the Children, 2019) these contexts are also the least likely to have the human resources and infrastructure 
needed to use EdTech effectively and at scale (Dreesen et al., 2020). 


This report aims to add to the growing evidence base on the use of EdTech in low-resource settings. In 
particular, the report presents findings on two until now relatively sparsely researched topics: The use and 
effectiveness of EdTech in a protracted humanitarian context (for Syrian refugees in Lebanon) and the transition 
to, and effectiveness of, EdTech for remote learning during COVID-19. More specifically, this research focuses 
on the implementation of the Akelius Digital Language Learning Course (subsequently referred to as the digital 
course) in NFE classes in Lebanon that serve Syrian refugee children. 


Box 1. The UNICEF-Akelius Foundation Innovation in Education Partnership 


In 2017, UNICEF Headquarters, UNICEF Sweden and the Akelius Foundation engaged in a partnership 
1Co M10] 0) fe) da Ol N\| | ©) = wall mo [U[ox~ Ht) aM alm tl aal-yge(-lale)(=\om (i) =) Me) cele] e-Vaalaalcise 
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starting in Greece in 2017 and subsequently expanding to Lebanon, Mauritania, Bosnia and 
Herzegovina, Serbia and Italy. The digital course, used on tablets, mobile phones or computers, is a 
iKole) Ko Ms{0|e) eco) auml- lave] 0l-le[om (cr-]aallave mel anve)alemacviUle(-\ocormaallele-lalecwmr-) ace (lavelel (sytem anliarelaiu(ctom-]qel0 lave muat-myvield len 
aiatewer-lagat=yesiall om-liaaksmcomals)|emeiall (eleven (cr-]aam (ciate) 0l-le-xsmcomr-loll [uecitommalolim fare lUlsjlolam | alco mye s(ole) (smal alersit 
communities and societies more broadly. 


Within the partnership there is a strong emphasis on evidence generation to track progress, learn, 
inform, and improve implementation. In each context, resources are invested to understand how 
anyelt=\aakcialesidcelameymuat= mello lite] MKcr-laall are mexolU] gs\-Mm AZo) a <smn (on V\c-1e0\smlan] eco) alae m(=x-leallaremcolmreial|(ela-iamelare 
adolescents. This research is part of a series of reports examining the use of the digital course for 
children’s learning — the first report explored the co-creation, use and effectiveness the digital course in 
(Cigqxever-m(.<-1r-]00) el-)ale(elUm-ar-] uyA0PA0) F 


Unlocking Learning 


The implementation and effectiveness of digital learning for Syrian refugees in Lebanon 


1.1. Education for Syrian refugees in Lebanon 


Driven by the influx of Syrian refugees into the country, the population of Lebanon rose from 5.2 million in 
2011 to 6.9 million in 2019 (World Bank, 2021). An estimated 1.5 million Syrian refugees are present in Lebanon 
(UNHCR, 2020). This increase in population has put great strain on the Lebanese education system, as 40 per 
cent of school-age children and adolescents in Lebanon are Syrian refugees (UNICEF, 2021). To respond to this 
crisis, the Ministry of Education and Higher Education (MEHE) has undertaken multiple initiatives to provide 
Syrian refugee children and adolescents access to education. These efforts include waiving primary school 
fees and offering a second afternoon shift in public primary schools to increase education access for Syrian 
children (de Hoop et al., 2018). However, even with these policies implemented, more than half (57 per cent) 
of Syrian refugees remain out of education in Lebanon (Human Rights Watch, 2021). 


Refugee children in Lebanon face numerous challenges to access education and achieve quality learning. As 58 
per cent of Syrian refugee households live in extreme poverty the opportunity cost of attending school can be 
very high (El-Ghali et al., 2019). Distance to schools, and the complex transportation system to bring children 
to schools, discourages enrolment and attendance, especially for girls (Human Rights Watch, 2016). In many 
cases there is a lack of awareness and knowledge of education opportunities and the necessary procedures to 
enrol children into schooling. Administrative barriers to enter into school remain, from specific documentation 
such as proof of residency that families need, to stringent and short school registration timeframes (ibid)'. In 
many cases, Syrian refugee children have faced long periods of education disruption and suffered significant 
learning loss, making it challenging for them to return into schooling. Once in school, Syrian children are also 
more likely than their Lebanese peers to experience violence, bullying and harassment (E!-Ghali et al., 2016). 


A significant barrier to Syrian children accessing and succeeding in schools is Lebanon's trilingual education 
system. While Arabic is the main language of instruction from early years, English or French (the choice of 
which depends on location) is taught in classes from primary school and used as the language of instruction 
for maths science classes (Jalbout, 2015). This system puts Syrian refugee students who come from a single 
language system without a strong understanding of English or French at an extreme disadvantage, especially 
in the transition to secondary schooling. 


1.2 Non-formal education as a bridge to formal learning 


To provide a pathway to bring out-of-school children into formal education system, the MEHE has developed 
the Reaching all Children with Education (RACE) five-year plans (RACE | 2012-2017 and RACE || 2017-2021) 
(Crul et al., 2019). These plans rely on local and international non-governmental organizations (NGO) partners to 
implement non-formal education programmes for Syrian refugee children and adolescents that are regulated 
by the MEHE. Non-formal education programmes run by NGO partners across the whole Lebanese territory 
work in a coordinated fashion to support children’s learning close to their communities (Shuayb et al., 2014). 
Each programme is tailored to respond to the specific needs of students at different age groups and learning 
levels: 


= Community-Based Early Childhood Education (CB-ECE). Young children starting from age 3 to age 
6 are provided with structured play and learning opportunities focused around six domains of child 
development — Arabic language, maths, science, social-emotional, physical/psychomotor and artistic 
lessons, psychosocial support, and introduction to foreign languages (either French or English) to ensure 
they are ready to transition to primary school. 


= Basic Literacy and Numeracy (BLN) and Youth BLN. Children and adolescents aged 10-14 years with 
no, or very limited, education experience are provided the foundational skills needed for the transition 
into formal schooling. BLN graduates’ transition either into the Accelerated Learning Programme or 
directly into formal education depending on their level and age (see below). The Youth BLN programme 
supports older adolescents and young people aged 15-20 years with little, or no, education experience, 
and follows a curriculum adapted to this age group that includes life skills and competency-based 
learning to support employability. 


1 Although the school year in Lebanon starts in early August and ends during late May of the next year, school registration closes in 
February with no more children admitted afterwards. 
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= Accelerated Learning Programme (ALP). This provides primary school equivalence using an 
accelerated curriculum for Grades 1-9 and is a direct bridge for children and adolescents into formal 
schooling. Children who have missed two years of schooling or more, including those who completed 
the BLN, are able to join the programme from the age of 7. 


= Technical-Vocational Education and Training (TVET). Vocational training programmes conducted by 
NGO partners are available for adolescents aged 15 or older to prepare them for employment. 


1.3. The introduction of the digital course in NFE classrooms 


In 2019, three NFE partners supported by UNICEF Lebanon, the Ana Aqra Association, Mouvement Social, 
and the Lebanese Organization for Studies and Training (LOST), began testing the use of the digital course for 
foreign language teaching (either English or French) in their classes. The course was introduced in a CB-ECE 
centre in Beirut, run by Ana Aqra, and two centres providing BLN classes, run by Mouvement Social in Mount 
Lebanon and LOST in the Bekaa Valley, respectively. The choice of either the English or French version of the 
course was determined by the NFE partner based on the language of instruction of the public schools in their 
respective communities. For the centres in Beirut and Mount Lebanon the English version was used, while in 
the centre in the Bekaa valley the French version was used. 


Image 2. Digital course content 


= Akelius 


language > francais > A2 > Comment te sens-tu ? 


Comment te sens-tu ? 
Sw 


lecture game guessing song 
= 
— = 


In 2019, the digital course was introduced to classes in a blended teaching and learning approach. This meant 
that the digital course was used as a teaching tool within a traditional face-to-face classroom implementation. 
Feedback from teachers in Lebanon indicates that there were three general steps teachers took when 
introducing the digital course into their classrooms. First, teachers explored the functionality of the digital 
course and its content, identifying how it could be used in their classes and began to plan how to integrate the 
tool into their lesson plans. Second, teachers began to introduce the digital course slowly to students in class, 
initially as a fun activity or as a reward. Third, teachers undertook more systematic integration of the course into 
their regular teaching practice and lesson plans, to reinforce and supplement learning done through traditional 
methods (this is consistent with findings on the introduction of the digital course from Greece (Karamperidou 
et al., 2020)). The content and features of the English and French versions of the digital course were developed 
iteratively using a co-creation approach, where teachers and UNICEF education staff provided regular feedback 
to the software development team based on their use of the course in classes with students’. 


2 This co-creation approach is explored in more detail in Karamperidou et al., 2020. 
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In March 2020, due to the COVID-19 pandemic, all face-to-face classes were suspended in Lebanon and NFE 
partners transitioned to remote learning. During this transition, the use of the digital course expanded from 
three learning centres to more than 60 centres as it was used by NFE partners as a tool for remote learning. 
The remainder of this report explores findings related to three key time periods: 


1. The implementation of the digital course in a blended teaching and learning approach prior to education 
centre closures and its perceived effects on learning in classrooms (June 2019—Feb 2020); 


2. The transition to remote digital learning for NFE partners in Lebanon (March 2020—August 2020); and 


3. The effectiveness of digital remote learning during COVID-19 on students’ learning (August 2020- 
November 2020). 


©War Child Hollan 
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2. Methods and research objectives 


This report relies on a combination of quantitative and qualitative methods to understand the process of 
implementation and effectiveness of digital learning for Syrian refugees in Lebanon. Quantitative data is used 
to understand the transition to remote learning and the associations between various factors including the 
use of the digital course and children’s learning outcomes. Qualitative data collected from July-August 2020 is 
used to understand the process of implementation of the technology both in person and remotely and to gain 
further insight into the potential pathways of results found in the quantitative analysis. 


2.1. Quantitative data sources and analysis 


The quantitative analysis in this report relies on two sources of data. 


The first source of quantitative data is an administrative NFE database that includes children’s academic and 
demographic information collected through the Child Level Monitoring (CLM) platform. The CLM platform is a 
tool to monitor the education progress of every child enroled in UNICEF Lebanon-supported NFE programmes. 


The CLM designed and managed by UNICEF Lebanon is also a monitoring tool to inform referrals to cross- 
sectoral support and services. The database includes basic demographic information on NFE participants as 
well as results from regular learning assessments that are collected at the beginning and end of each round 
of the respective NFE programme (including but not limited to CB-ECE and BLN). For both CB-ECE and BLN 
students, the majority (more than 97 per cent) are Syrian, and 51 per cent of the sample are boys. These basic 
demographic characteristics do not differ between centres that adopted or didn’t adopt the digital course during 
2020 (see Annex A)’. There are 10,711 children in the dataset who attended CB-ECE classes remotely, with 
4,745 children studying at 26 learning centres that reported using the digital course in their remote learning 
programme. For BLN, there are 4,354 students in the database, and 2,492 children study at 38 learning centres 
that reported using the digital course during this NFE round. 


Learning assessments are collected across seven domains for CB-ECE (Arabic, foreign language (French or 
English), maths, art, social-emotional, psychomotor, and science) and four domains for BLN (Arabic, foreign 
language (French or English), maths, and social-emotional). Scores on the assessments are on a 0-100 scale. To 
understand the effectiveness of remote learning during 2020, the learning assessments collected at the start 
and end of the NFE round that took place from August-November 2020 are used. 


The second source is a short survey sent to education staff from NFE centres around the country to learn 
about the transition to remote learning. This survey encompassed NFE centres that are UNICEF partners 
as well as centres that are not affiliated with UNICEF. In total, 72 learning centres from 10 organizations 
responded to the survey in June 2021. 


2.2. Qualitative data collection 


The study also draws from qualitative data collection and classroom observations in the three NFE learning 
centres where the use of the digital course was piloted (see Annex Table B.1). 


Between 23 July and 10 August 2020, in-depth interviews were carried out with the head of each of the three 
initial partner organizations, education coordinators who manage the learning centres where the digital course 
was introduced, and the Akelius software designers and programme managers. Focus group discussions were 
conducted with teachers and students. 


Due to COVID-19 restrictions, qualitative data collection was partly done remotely (via Zoom or Skype). A 
few online classroom observations were carried out during the closure of the learning centres. Resulting 


3 Except for the differences in the type of caregiver (either the mother or the father), no significant difference was found in terms of 
children’s demographics according to whether their learning centres used the Akelius Digital Language Course. 
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observational data on the use of the digital course as a remote learning tool were used for validation and 
triangulation purposes. In total, 34 individuals were interviewed. 


Data collection, storage and management were in line with the UNICEF Procedure on Ethical Standards in 
Research, Evaluation and Data Collection and Analysis. Research instruments and fieldwork protocols were 
approved by the Health Media Lab (HML) and the Institutional Review Board of the Office for Human Research 
Protections in the US Department of Health and Human Services Research. To code, organize and analyse 
qualitative evidence the research team employed thematic content analysis. 
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3. Findings 


3.1. Pre COVID-19: In-class use of the digital course and perceived effects 


Teachers introduced the course in different ways to fit it into their 
practice. While students in each learning centre had individual tablets 
to use the course, in one of the selected learning centres, teachers 
routinely displayed the digital course on a projector, allowing learners 
to make connections between the content they encountered in the 
course and the lesson all at once. Some teachers used time for students 
engaging in the digital course on tablets for reviewing key concepts at 
the end of the class, while others used it as a core teaching method, 
as they found that its increased learners’ engagement, reducing the 
risk of dropout. 


“Tl use it for] everything 
related to new words, 
sentences, how to introduce 
yourself, your family, how 

to express yourself, and 


your feelings. It helped the 
children a lot, especially on 
everything related to oral 


Education coordinators confirmed that the digital course was a olnalanlelalic:-lelolae 
flexible tool that teachers used in various ways to achieve learning 
goals, depending on their lesson plans and the themes they wanted 
to introduce. However, the importance of consistent teacherlearner 
interaction supporting what was learnt on the digital course was seen 
as equally crucial. 


— Teacher 


Teachers saw improvements in classroom management. Participants 
from each of the three learning centres commented on the positive 
impact of the digital course on the classroom environment. Teachers 
mentioned that students appeared happier, as the digital course 
gave them the opportunity to explore letters, words, and numbers 
in an interactive manner, using games, images, and multimedia. This 
made language and related concepts more meaningful and turned the 
learning process into a fun experience. The digital course was largely 
perceived as encouraging active learning where students are involved in 
the learning process and not passively listening to a lecture. Educators 
— Education Coordinator further explained that the increased difficulty levels for games, and the 
instant feedback and scoring that the course provides, were particularly 
useful in keeping students disciplined, motivated, and engaged. 


“Students may memorize 
words from a game but this 
doesn't mean that they can 
write them down. Worksheets 
iie)mne)e-\oillexomr-]ale Moi) 0) elo)amicelan 


the teacher are essential 
to solidity newly acquired 
knowledge.” 


The digital course encouraged self-paced learning and the 
individualized monitoring of students’ progress. Interviewed 


teachers and education coordinators indicated that students, especially “There is excitement, 

those at lower learning levels, benefited from the personalized and self- competition. It changes 
paced use of the digital course. Learners could decide for themselves atcmaaleveleRoumialenelecratat 
how fast they moved through the content and whether they repeated learning process itself in class 


activities. has improved. The scoring 


This self-paced learning is difficult to achieve in group instruction and MUMEUOn creates \compeutive 


teachers suggested it increased students’ autonomy and engagement 
in the learning process, facilitating their learning. 


eye} caliatem=vanVdlne)alant=laver-laavelare) 
students that pushes the 
overall performance of the 
class.” 


Patol Ucersiaelsmexele)aeliit-icelg 


15 


Unlocking Learning 


The implementation and effectiveness of digital learning for Syrian refugees in Lebanon 


The self-paced nature of the digital course made students feel equal and 
more confident, which teachers mentioned was especially beneficial 
“By allowing children to for children with disabilities and learning difficulties. Respondents also 
see their own progress, explained that the digital course helped teachers monitor the progress 
[the digital course] helps of their students individually. Teachers could see the progress of each 
them realize that they are learner to get a sense of how much they had grasped of a concept, to 
achieving something they determine whether they were falling behind, and to provide individualized 
could not do yesterday.” support when necessary. 


-Teacher Teachers perceived that the digital course increased students’ 
speaking skills and confidence. Teachers observed that the in-class use 
of the digital course improved students’ communication skills, especially 
their capacity to speak clearly and confidently. Learners confirmed that 
the repetition of words and linguistic exercises made them more confident and less self-conscious when 
speaking a new language. Interview data further suggest that the use of the digital course helped students 
identify their strengths and weaknesses individually and to correct their own mistakes. For many educators, 
this was especially evident with younger students in the CB-ECE programme, who appeared more hesitant to 
speak and to make mistakes during class. 


3.2 Transition to digital remote learning 

In response to the COVID-19 pandemic, in March 2020 all schools and NFE partners closed in-person classes. 
This prompted NFE partners to quickly prepare for remote learning. Qualitative data collection from three 
initial NFE partners that implemented the digital course and a survey of 72 NFE learning centres covering 


responses from 10 separate implementing partners illustrate that the transition period to remote learning 
followed multiple steps. 


Figure 1: Frequency of contact with parents 
1% 


BB Daily 

si 2-3 times a week 
1) Weekly 

| Monthly 


Source: NFE Short Survey (June 2021) 


3.2.1. Reaching out to families and starting regular communication 


Qualitative evidence from each of the three organizations showed the importance of staying in close 
communication with learners and their families. One education partner began connecting with parents on a 
weekly basis starting with psychosocial support activities. Another partner began by having teachers contact 
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all previously enroled children on WhatsApp, teachers then created 
WhatsApp groups for students by learning levels. 


From the survey of 72 NFE learning centres, 64 per cent reported that 
they had kept in touch with caregivers daily throughout the period of 
centre closures, around a quarter were in contact two or three times a 
week, and a 10th were in contact once a week (see Figure 1). Keeping 
a strong relationship with the refugee community was extremely 


“We started with an 
awareness session and built 
close relationships with the 
community. The parents were 


very receptive and it was a 
good time they spent 


important, as NFE partners noted that distrust of institutions made with us.” 


refugee families initially skeptical of remote learning programmes. SNEEeducationcoordinator 


3.2.2. Development of remote lesson plans while curating 
digital content 


Planning for remote lessons and curating content was the second key step in the transition. One partner 
explained that they split their teachers into groups. Each group was assigned a specific learning goal and was 
tasked with creating and curating video content aligned with those goals. Initially, the goals were focused on 
awareness around health and safety during COVID-19. Gradually, topics shifted towards learning goals aligned 
with the NFE curriculum. 


Figure 2 shows the modalities of communication and the types of content used for remote learning by NFE 
partners as of August 2020. WhatsApp was used as a method of communication by almost all learning centres 
(97 per cent). The digital course was used at 68 per cent of centres reporting, and 39 per cent of learning 
centres reported delivery of printed materials to students. 


Figure 2. Modalities of communication and content used for remote 
learning 


WhatsApp Akelius Delivery Zoom YouTube Other 
language of printed digital tools 
program materials 


Source: NFE Short Survey (June 2021) 
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3.2.3. Delivery of devices, connectivity, and guidance on using digital devices and platforms 


A major challenge revealed from both the qualitative interviews and the broader survey of NFE centres was 
unstable electricity and connectivity for refugee families. Around 69 per cent of surveyed centres listed 
internet connectivity of students as a major challenge in remote learning (see Figure 3). Similarly, 31 per cent 
of learning centres reported that teachers faced internet access issues as they taught. While educators were 
curating content in the transition to remote learning, they were at the same time working in communities to 
help address these technical barriers to accessing remote learning for their students. 


The education partners reported providing internet Wi-Fi hotspots, subsidized internet cards and internet top- 
ups to their students. As access to the necessary technology does not ensure take-up, educators stressed 
the importance of regular support to families to effectively use technology. An education coordinator from 
one partner stated that even though some students had previously used the digital course in class, in this 
new context of remote learning education staff acted as if everyone was new to the programme. In some 
cases, teachers set up a step-by-step video on how to download, sign into the course and use it online and 
offline. In other cases, field officers visited communities and helped parents and children with technology, 
from downloading the application to installing internet routers and updating content using the ‘Mesh Net’ 
function. The ‘Mesh Net’ function, created by the developers, allows one device (tablet or mobile phone) to act 
as a server to transfer content updates to other nearby devices without internet connection. 


Figure 3: Challenges faced by learning centres during transition to remote 
learning 


Internet Communication Internet Reaching out Monitoring 


connectivity with parents connectivity to children of students’ 
of students of teachers progress 


Source: NFE Short Survev (June 2021) 


3.2.4. Remote teaching using WhatsApp and Zoom as communication platforms 


Remote classes were commonly conducted using Zoom and/or 
WhatsApp groups. The choice of using Zoom, WhatsApp, or acombination 
of the two depended on a number of factors. In general, caregivers 
were more familiar with WhatsApp and more willing to let their children 
use it. Some parents were hesitant to allow their children, especially 
girls, to use Zoom with the video camera on. In areas where internet 
connectivity wasn't stable enough to conduct classes over Zoom, NFE 
partners would set up WhatsApp groups of three students to deliver 
lessons, however this was extremely time consuming for teachers to — BLN Student 
organize and carry out. Across each of the partners, WhatsApp was the 


“The teacher sends us links 
to videos; we open them 
and send coursework back. 
We also watch videos on 


YouTube. We regularly use 
YouTube, Akelius, Zoom.” 
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tool most commonly used for coordination, sending links to videos and other resources, scheduling classes in 
group chats and receiving feedback individually on work completed by students. 


Experiences in implementing remote teaching and learning varied across learning centres and the challenges 
faced in different implementation settings. Box 2 summarizes various experiences of different remote classes, 
based on observations of virtual classes and interviews with education coordinators (conducted in July 2020) 


As noted in Box 2, there are clear variations in remote teaching based on the characteristics of learning 
centres, educators, communities, and students. There is also suggestive evidence that there were systematic 
differences in the implementation of remote learning modalities and tools between centres that chose to use 
the digital course and those that did not. Figure 4 illustrates these variations. 


Box 2. The use of the digital course in three different remote classrooms 


Class 1: Practicing using the 
digital course in the remote 
classroom and reviewing 
progress after each step. 


The teacher started the virtual 
class conducted over Zoom by 
ojatsvordialemoluncialelclalexomalare mantel dlave) 
sure that all students had access 
to the digital course. He then 
asked students to go to Level 8 
late icom-\ahe)mudal=limUls\>)aarelaatsmclale| 
password. 


As they started, the teacher 
ankeldaatqtomsyiUlel-Valecmaat-landalen, 
should send a picture on the 
VAVakclicy-Neyeme]celulomeymaarcyis 
progress as they completed 
each step. 


While the teacher viewed the 
photos, he provided feedback. 

If any delay or mistakes were 
noticed, he addressed individual 
students separately and muted 
all others in order to provide 
tailored feedback on how to 
complete the exercises. 


Source: Observational data 
(ole) | [=Xe1t=1o ml WLUI\APAOPAO) R 


Class 2: Using the digital 
course as homework following 
a remote class. 


As a follow-up to the remote 
learning class, the teacher asked 
students to practice writing 
sentences in their workbooks 
and to send pictures of their 
work using WhatsApp. 


iilatemicr-\e1alolar-] Sem-alerel0]r-le(-1e 

students to practice using the 
(of{opir=| mevolUlest-me) am ualclime)mials lie 
caregivers’ phones. 


The digital course was not used 
in class. Instead, the teacher 
(o[U]lo lero MsidU(o(=Valesmnomere)anle)(=1K=) 

a certain level in the course 

io o}g-\o1u(exomuaromce)e)ron-laremaals 
technique she had taught during 
class. 


After finishing the session, 
students shared screenshots of 
their work on the digital course 
on the WhatsApp group. 


Source: Observational data 
(oro) |[=York=1o (6/01 APAOPA0) B 


Class 3: Ad-hoc use of the 
digital course leaving it up to 
the families. 


The centre coordinator stated 
that after seven weeks of 
experience with remote 
learning, teachers still needed 
a better plan to implement 
uakomeolle]it-| meolUl esto mw aiaalianmdalcyia 
remote classes. 


Parents in that community 
were having trouble adapting 

to remote learning. As a result, 
the team was more focused on 
staying in regular contact with 
families via WhatsApp, leaving 
li g\oMUls\o Moya taYomel(e] ite] Merl] Exo M0] ©) 
to the parents and the students. 


Source: EC Interview (July 
VAOA0) 
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Figure 4: Communication modalities and digital content used in remote 
learning, by digital course adoption 


No digital course Digital course 
(N=24) (N=48) 
100% 
fe) 
46% AA% ‘ 
38% 
29% 
25% 

8% 6% 

WhatsApp Zoom Delivery YouTube Other Other WhatsApp Zoom Delivery YouTube Other Other 
of printed digital non-digital of printed digital non-digital 

materials tools tools materials tools tools 


Source: NFE Short Survey (June 2021) 


Centres that adopted the digital course as part of their remote learning curriculum were more likely to report 
using any type of digital tool/content in the survey of NFE centres conducted in June 2021. In particular, 
centres that adopted the digital course were far more likely to use Zoom (46 per cent vs O per cent) and 
to share content with students, such as YouTube videos (38 per cent vs 0 percent). This provides a strong 
indication that centres that opted to use the digital course were already better able to transition to remote 
learning, based either on their capacity, community access to digital devices and connectivity, or likely a 
combination of the two. 


Learning centres that did not adopt the digital course reported higher rates of internet-related challenges, both 
for teachers and students (see Figure 5). While 79 per cent of learning centres that did not adopt the digital 
course reported internet connectivity as a challenge for remote learning, 65 per cent of learning centres that 
did adopt the digital course reported the same issue. 


Figure 5: Challenges faced by learning centres during remote transition, by 
digital course adoption 


No digital course Digital course 
(N=24) (N=48) 
79% 
oe 29% 
25% 25% 23% 
a 8% 9 17% 
4% 
a | CU 
Internet Communication Internet Reaching Monitoring Other Internet Communication Internet Reaching Monitoring Other 
connectivity with connectivity out to of students’ challenge connectivity with connectivity out to of students’ challenge 
of students parents of teachers children progress not listed of students parents of teachers children progress not listed 


Source: NFE Short Survey (June 2021) 
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A higher proportion of centres that adopted the digital course reported communication with parents as a 
challenge. While more investigation is needed, one hypothesis is that areas with the digital course had higher 
connectivity. Thus challenges that occur during the use of digital learning including engagement with parents 
become more salient than in areas where connectivity is a hard barrier. 


3.3 Effectiveness of Digital Remote Learning 


This section investigates children's learning outcomes in the first semester or round of remote NFE classes 
from August-November 2020. The analysis first explores how children learned overall during remote learning 
in the CB-ECE and BLN programmes. Second, the analysis investigates the outcomes of students in centres 
that adopted the digital course within their remote learning programmes and those that didn't. 


Descriptive analysis of learning outcomes from each of the NFE partners outlined in the following section 
shows that the centres adopting the digital course had higher baseline learning assessment scores across 
domains. Therefore, a number of controls are used in the econometric analysis in the following section to 
reduce the risk of omitted variable bias. 


3.3.1. Overall learning gains during remote classes 


Students attending both CB-ECE and BLN classes remotely improved learning outcomes across all domains 
measured in the CLM database (seven domains for CB-ECE and four domains for BLN)?. 


Table 1 presents a summary of learning assessment scores at the beginning and at the end of the semester 
for students in the CB-ECE and BLN classes. For the foreign language (French or English) competency of CB- 
ECE, students improved scores by an average of 43 points (73-30). A similar increase of 34 points (58-24) is 
observed in the same competency for older BLN children. 


Table 1: Competency scores before and after the first semester of remote 
learning for CB-ECE and BLN (out of a scale of 100 points) 


Foreign Social 


FValeperele(: emotional 


N Pre Post Pre Post Pre Post Pre Post Pre Post Pre Post Pre Post 


CB-ECE | 10,711 30 73 35 78 32 79 45 86 4 81 48 85 36 83 


BLN 4,353 24 58 37 68 35 69 = 52 77 = = 


Source: Child Level Monitoring (CLM) database 


4 See Annex C for assessment tools and competencies assessed at learning centres. 
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3.3.2. Use and effectiveness of the digital course in remote learning 


Among the seven domains of learning recorded through the CLM database for CB-ECE students, analysis on 
the influence of the digital course on learning focuses on foreign language, Arabic, maths and art (see Table 2). 


Table 2: Competency scores before and after the first semester of remote 
learning for CB-ECE, by digital course (100 point max.) 


Foreign 
Vite Lt rve(: 
N Pre Post Pre Post Pre Post Pre 
No digital course 5,966 19 59 29 68 25 72 36 78 
Digital course 4,745 43 91 43 91 4 89 56 96 
Across 10,711 30 73 35 78 32 79 45 86 


Source: Child Level Monitoring (CLM) database 


The choice to focus on these competencies is based on the content and features of the digital course, which 
teaches foreign languages (English or French) and has a heavy focus on songs and games to enrich interactive 
learning. Although the digital course does not have Arabic content, studies such as Kaushanskaya el. al., (2011) 
show associations (both positive and negative) between first and second language acquisition, suggesting that 
second language experiences influence first language learning. Therefore, the analysis considers the potential 
correlation between second language acquisition and native language learning. 


Children studying at the learning centres that adopted the digital course performed better across all 
competencies at the time of the baseline learning assessment in August 2020 than children in the learning 
centres that didn’t adopt the digital course. In terms of foreign language, children attending remote classes run 
by centres that used the digital course scored 42.5 points at the beginning of the semester, while children in 
centres where the digital language course wasn’t used scored just 19.3 points. Given these initial differences 
in competencies, and the self-selection of centres to use the digital course, the analysis is careful in the 
examination of the effectiveness of the digital course, ensuring that the baseline differences in competencies 
scores are controlled for. 


The analysis finds that students taking the digital language course performed better on learning assessments 
in foreign language, Arabic, and art than their peers who didn’t do the digital language course. These results 
are statistically significant and control for students’ individual characteristics; the way the assessment was 
conducted; students’ baseline learning levels determined through a pre-test, and geographical differences in 
the learning centres (see Box 3 for a complete explanation of the empirical procedure). 
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Figure 6 shows the estimated difference in scores for CB-ECE students in centres that adopted the digital 
course compared to those that didn’t, on each of the four learning competencies of interest after controlling 
for pre-semester learning levels, demographic characteristics and the location of implementation. Children in 
centres that used the digital course as part of their remote learning programme show a statistically significant 
improvement in foreign language, Arabic, and artistic scores. The foreign language scores of children exposed 
to the digital course is 21 points higher (0.7 SD) than children studying at the learning centres that didn’t 
introduce the digital course. Similarly, Arabic and art competencies increased by 21 points (0.9 SD) and 11.2 
points (0.5 SD), respectively®. Further analysis shows that there is no significant difference in effect by gender 
(see Annex Table D.2). 


Figure 6: Estimated improvements in learning competencies associated with 
digital course use for CB-ECE children 


40 4 


yi b=21.2** Ab=21.1*** 


‘e b=11.2*** 
- b=6.3 


Digital course 


e Foreign language a Arabic = Maths x Artistic 


Source: Child Level Monitoring (CLM) database 


Notes: “b” in the figure represents beta coefficient. 
The asterisk shows the statistical significance of the coefficient: *p<.10, **p<.05, and ***p<.01 


5 To consider the non-linear effect of the digital course on competencies, scores standardized to standard deviation are also presented 
(see Annex Table D.1). 
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Box 3. Empirical estimation strategy and robustness checks 


To estimate the association between the use of the digital course during remote learning and learning 
outcomes of students, the empirical analysis was conducted using the following equation: 


Post assessment,= a + B, Digital Course ,+ 0 Pre assessment, + X,+ VY, + P, + €4 


The primary variable of interest is the post-semester assessment score (Post assessment.) and the 
difference (B,) between those attending classes with the digital course and those without. 


To isolate the association with the use of the digital course we control for a number of centre-level (c) 
or individual-level (i) characteristics. X,, is a vector of students’ individual characteristics: gender, age, 
disability status and exposure to psychosocial support programmes. These characteristics are included 
io oxo)aha xe) mie lmraleico)acwiar-) alaate\allanilOl-laler-melliicl-larel-somlalme:ay| (el colaksm lato] aallarem (n’{c) sem =e [Ul=]|NVall an) ele) at] alamo) 
Pre assessment, controls for individual students’ pre-semester learning levels using pre-test scores 
taken in August 2020. As the decision to take up the digital course was up to each learning centre, it 

is likely that unobserved characteristics are correlated with both the choice to use the digital course 
and the children’s learning outcomes. Based on the assumption that those unobserved characteristics 
would affect the pre-semester and the post-semester scores in a similar way, the analysis uses 

the baseline score as a control to reduce the risk of omitted variable bias. In this sense, the results 
presented in this section focus on performance improvement over the semester. Finally, the estimation 
controls for geographical differences by including learning centre fixed effects (y,) and the way in which 
the assessment was conducted, as this varied based on the situation of the pandemic in each locality 
(e.g., online assessment, assessment over the phone, in-person visit) (9,)'. The standard errors (€,) are 
clustered at the learning centre-level as the exact implementation of remote learning is determined by 
learning centres. 


To check the robustness of the results, the analysis on maths is presented as a false outcome test. 
This is a ‘false outcome test’ as the research doesn't expect a significant association with maths 
results from use of the digital course. Although the recent version of the digital course offers some 
maths content, this content is far less developed and teachers during the period of analysis reported 
not having used maths content in their classes. Therefore, results showing a non-significant effect 
for maths scores support the hypothesis that the main findings on languages or art aren't driven by 
external factors other than the use of the digital course. The same estimation was conducted on 
other competencies that have fewer tangible links to the content of the digital course namely social- 
emotional, psychomotor, and science (see Annex Table D.3). The expectation is that, since these 
outcomes should not be directly tied to the content and features of the course, one would expect 

a more muted association between the use of the course and these outcomes. Results from these 
additional false outcomes checks are similarly non-significant. 


1 Note that no significant differences between scores are observed based on how the assessment was conducted (online, over 
the phone, or in person). 


For BLN students, competencies on foreign language, Arabic, 
maths, and social-emotional development are collected and stored 
in the CLM database. Running the same model, improvements in 
foreign language scores were observed but the results were not 
statistically significant (see Annex Table D.4). This different result 
from the CB-ECE findings may highlight that CB-ECE children more 
quickly gain language skills as early exposure to a second language 
is related to higher proficiency of the language (Krashen et al., 1979: 
Singleton, D., 2001). It is also important to note that the sample size 
used for the CB-ECE analysis is twice that of the BLN. — Student 


“The nice thing is that it [the 
digital course] includes many 
pictures and photos. You can 
give me a new word for a month 
but | will never learn it without a 
picture. If a picture is attached to 
the object, | can remember the 
word for it.” 
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Qualitative evidence corroborates quantitative findings, with teachers reporting perceived increases in 
language learning outcomes across both BLN and CB-ECE programmes. Educators noted that listening was 
the language skill most improved in remote learning. When working on the digital course, learners could listen 
to specific words and phrases multiple times at their own pace. This 
seems to have improved word recognition and English and French 
pronunciation. Students were happy that they could log in to the 
course independently and felt that the visual cues embedded in the 
application were enhancing their learning experiences, facilitating 
— Student their understanding of objects (e.g., the tomato) and their capacity 
to read and write short words and sentences. 


ma Cole lalr-l(ola\-elamuatome lle] ii] 
(oXolUlacX- Pani MalolUnur-lanyelaromare)| eliare| 


me. | learn a lot.” 


3.4. Effects beyond students’ learning 


In both the survey of NFE centres as well as qualitative interviews education coordinators mentioned 
externalities (or spillover effects) from remote learning. In certain areas education coordinators reported that 
attendance at remote classes was higher than for in-person classes, noting that students that live in harderto- 
reach areas, or older students who work, were more able to attend. However, the challenges to engage with 
children and families from a distance and the connectivity issues across Lebanon should not be underestimated 
(as examined in section 3.2). Education coordinators also reported that in many cases parents took part in 
classes with their children and learned from the digital course. As closures made constant and continuous 
engagement between learning centres and parents a necessity, educators observed that this led to their 
increased engagement in their child’s education. However, they also explained that there is no substitute for 
in-person learning, and the type of care, feedback, and interaction that teachers provide to students and that 
students have with their peers in the classroom. 


©War Child Hollan 
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4. Conclusions and recommendations 


The closure of in-person classes around the world as a response to COVID-19 has shown the lack of preparedness 
of education systems to continue children’s learning from a distance. Developing digital learning systems and 
practices that can be leaned upon and used in times of crisis could also enrich education practices in times of 
normality. However, for digital learning systems to be effective they must be inclusive, providing opportunities 
to accelerate children’s learning, rather than entrenching existing inequities (Brossard et al., 2021). Those who 
stand to benefit the most from digital learning are also those with the least access to the tools and support 
needed for digital learning to succeed. 


In the case of Lebanon, Syrian refugees face challenges related to poverty, discrimination, and lack of 
connectivity, all of which reduce their ability to succeed in education be that in-person or remote. As digital 
learning programmes and systems proliferate, there is an urgent need for the education community to test 
and learn from digital learning delivered in challenging circumstances, rather than piloting them in rich contexts 
with favorable connectivity and technology conditions. As noted in Unwin et al., 2020: “As a starting point, to 
deliver equity in the use of digital technologies in education means beginning where it is most difficult.” The 
experience of the UNICEFAkelius Foundation in Lebanon is one such example, implementing digital learning 
for those who could stand most to benefit under very challenging circumstances. This research from Lebanon 
adds to the growing base of knowledge showing that digital technologies can enhance marginalized children’s 
learning when technological barriers are overcome and the technology is implemented in a clear pedagogical 
approach supporting teachers and their practice. 


When the digital course was used during in-person classes, qualitative evidence from teachers indicates that 
the use of the course led to improvements in their classroom environment, student’s confidence, motivation to 
go to class, and speaking skills. When implemented in a fully remote learning environment, students from early 
childhood education classes that implemented the digital course showed statistically significant improvements 
in foreign language (French or English, 21.2 per cent increase), Arabic (21.1 per cent increase), and art (11.1 
per cent increase) competencies compared to students from centres that didn’t use the digital course, when 
controlling for pre-semester learning levels, location of implementation, and demographic characteristics. 


Teachers remained the key to successful implementation of digital learning when centres closed, providing 
extra-academic support to students and families. The role of teachers grew when classrooms closed, they 
provided psychosocial support to families, as well as technological support, from developing and delivering 
guides on the use of technology, to providing data package top-ups and even helping install internet routers 
within communities. In 2021, NFE education centres in Lebanon began to transition from full remote learning 
to a hybrid approach, with classes alternating between being delivered in-person and remotely to adhere to 
social distancing. Teachers are building on the lessons learned throughout the pandemic to better use digital 
tools in their classrooms, whether these classrooms are in-person or virtual. Key recommendations from this 
research relevant to governments, international organizations, and education practitioners seeking to build 
equity-based digital learning programmes to build resilience into education systems, are Summarized next. 
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ateXexe) aay aal=iaver-iarelalsmie) am ele) i (eavinat-].<-1 asym -le [0 ler-)ile)am e)e-leadi tle) all ace 
and EdTech developers: 


= Governments should prioritize investments in reliable electricity, and connectivity 
infrastructure, especially for hard-to-reach and marginalized areas and populations. 
At the same time, work must be accelerated with the private sector to ensure low-cost 
rTarem-laie)ael-le)(-Mel-it- paral MOlmYNd alle al-lgomalclere (oro mcomelar-]e)(-moxe [0 ]it-]e)(-mellelir-]micr-]aaliale pm axe [U(er-)aleyn) 
partners across Lebanon cited connectivity as the major barrier faced when providing 
(olfelicsl maclaatolccm (cr-lanllalepmere)anl efelelare(-lom onandalo walle] amelessi mo) me r-}i-F 


Frequent communication with families and communities should continue beyond 
remote learning to sustain increased caregiver engagement in their children’s 
learning. Education coordinators noted that caregivers were more engaged in their 
children’s learning while using the digital course at home. For refugee families with low 
trust of institutions, the trust and relationships built with community-based NFE actors 
(oxo}alulalUlctomicom eL- mola ii(er-] MKem=)alecel0le-lelomiclaallliotomcom elu] aciU[- mle 10 er-1t(o)ammielmcalclimeallieliclar 


Education providers must invest in the digital skills of teachers and facilitators and 
their readiness for digital learning, whether in the remote or physical classroom. 
Implementation of digital learning during school closures varied between centres, 
partners, and teachers. Many times differences were based on the capacities of teachers 
and various technological constraints in communities. Investments in the digital skills of 
teachers and facilitators need to be met with ongoing support and guidance for teachers 
and facilitators on how to effectively integrate technology into their teaching practice. 


The support system needed to enable digital learning must be invested in, 
and the key role of teachers in successful implementation of EdTech must be 
acknowledged and supported. Teachers provided extra academic support to families 
in alleviating technology and connectivity issues throughout school closures. The use 
of technology for learning requires much more than a device held by a student and an 
engaged teacher. Governments and education providers should cost and invest in the entire 
S10] 0) Le) m MESNVAsHK= 10 nino) Mme Ne] iK-Im(ctolaallare mate Ulollaremiamalelaat-laMccts1ol0lcexctom cone] l(nvdteltmccreralare) (ele |(ere]| 
and connectivity challenges, to monitoring and evaluation systems, and implementation 
research to understand how digital learning works. Doing this would reduce the burden 
that falls on individual teachers or organizations to provide technology support on top of 
their regular teaching duties. 


Yo) Aan te-Ta-Me(-\V@-1(0) oL-De-MdiLelei ce mel--1ie lam olaelele-Tiil-m-larem olatelaiar4--]oyeliter-hitelamict-laele-t 
that align with the needs of teachers and to the constraints of the learning 
environment. While interactive and engaging content should be a priority, it must be 
coupled with EdTech solutions designed to be used in low- or no-connectivity settings. 
Development of digital learning technologies that allow for interactive learning offline 
are critical to address the needs of the majority of children living in areas with no or 
intermittent internet connectivity. 
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Annexes 


Annex A - Demographic characteristics of children by the digital course 
use 


CB-ECE 


ome [fe The] Digital 
course couse 
Digital learning 


Individual information 


Age 5.54 5.31 5.82 4 6 
Male 0.52 0.52 0.52 0 1 
Nationality: Syrian 0.97 0.99 0.95 0 1 
Disability 0.04 0.03 0.06 0 1 
Caregivers 

Mother 0.17 0.25 0.06 0 1 
Father 0.83 0.74 0.94 0 1 
Other 0.01 0.01 0 0 1 
Observations 10,711 5,966 4,745 10,711 10,711 


Notes: The table provides the descriptive statistics of child demographic information 


No digital Digital 
course couse 
Digital learning 


Individual information 


Age 11.83 11.90 11.78 10 14 
Male 0.51 0.47 0.54 0 1 
Work 0.15 0.09 0.20 0 1 
Disability 0.05 0.03 0.07 0 1 
Caregivers 

Mother 0.40 0.38 0.42 0 1 
Father 0.59 0.61 0.57 0 1 
Other 0.01 0.01 0.01 0 1 
Observations 4,354 1,862 2,492 4,354 4,354 


Notes: The table provides the descriptive statistics of child demographic information 
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Annex B —- Sampling for qualitative data collection 


Table B.1: Selected learning centres for qualitative data collection 


afl) Gali Location Age of interviewd recipient Version of digital course 
Implementing partner 


Mouvement Social Ghobeiry 10-14 English 
LOST Bekaa 10 - 14 English and French 
Ana Agra Mount Lebanon 5-8 English and French 


Annex C —- Example of assessment tools for language and art development 
(CB-ECE) 
Child Progress Follow Up 
CB-ECE 
Level 3 


All the below criteria are related to the learning objectives of the curriculum of ECE. The ratings reflect the 
developmental level of the child in each domain at the time of assessment. 


Casa: 


Centre: 


Name of child or code: 


Name of facilitator: 


Date of the observation: 


Ratings: 


= N (Not observed): Child does not demonstrate skill or behaviour at this time 
= P (In Progress): Child demonstrates skill or behaviour with some support 


= O (Observed): Child can demonstrate skill or behaviour independently 


Note: During remote learning, this adapted assessment tool is used according to the colour coding below, and 
as soon as the direct implementation is back in centres — even if partially — then the original assessment tool 
is to be used. 
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Language Development 
Se 
a) 
Oral Language (Listening / Speaking) 

Follows three-step instructions 

Understands “What, Where, Who” questions 

Understands action verbs (stop, go, start, etc.) 

Speaks clearly and articulates words 

Identifies some letter sounds 


Retells information from a story 
Listens attentively and respond to stories read aloud 


ON OoaRWN 


Makes connections to own experience when listening to 
stories 


9 Speaks simple words and phrases to express needs, wants 
and feelings 


10 Uses new words learned in the unit 

11 Repeats rhymes and participates in singing songs 

12 Formulates more complex sentences of more than five words 
13. Tells own stories comprehensibly 


14 Uses comprehensibly high-frequency words (days of the 
week, etc.) 


15 Asks simple questions 
Written Language (Reading/Writing) 
16 Recognizes letters (lower case and upper case) 
17 Recognizes first name in print 
18 Recognizes own name and that of peers in the environment 
19 Shows interest in reading stories independently 
20 Reads high-frequency words 
21 Reads simple sentences of two to three words 
22 Uses scribbles, symbols, or drawings to express ideas 
23 Writes own name 
24 Writes some letters or words legibly 


25 Chooses to write as play, or in informal situations, for 
example, filling in forms, writing a menu or wish list for a 


party 
26  Differentiates between illustrations and written text 
Visual Language (Viewing / Presenting) 
27 Describes what is seen in a picture 
28 Recognizes symbols and logos in the environment 
29 Recognizes connections between pictures 
30 Sequences pictures to tell a story 
Subtotal for each rating 
Grand adapted total/60 


Comments: 
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Artistic Development 


Enjoys drawing and painting using available materials 
Adds details in artistic drawings 

Uses body, voice, and imagination to create characters 
Repeats simple patterns of rhythm and music 

Uses voice expressively 


aoa BWwWDN 


Utilizes voice and body as a mean of artistic expression 
Subtotal for each rating 
eSrcudedeneuioey iy 


Annex D — Robustness checks for empirical analysis and additional 
analyses 


Figure D.1: The association of the digital course on the main outcomes 
standardized to SD 


16] 
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Digital course 
e Foreign language « Arabic = Maths x Artistic 


Source: Child Level Monitoring (CLM) database 


Notes: “b” in the figure represents beta coefficient. 
The asterisk shows the statistical significance of the coefficient: *p<.10, **p<.05, and ***p<.01 
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Figure D.2: The effect of the digital course on foreign language for CB-ECE 
children by gender 
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Source: Child Level Monitoring (CLM) database 


Notes: “b” in the figure represents beta coefficient. 
The asterisk shows the statistical significance of the coefficient: *p<.10, **p<.05, and ***p<.01 


Figure D.3: The association of the digital course on the outcomes that do 
not expect the effects 
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Source: Child Level Monitoring (CLM) database 


Notes: “b” in the figure represents beta coefficient. 
The asterisk shows the statistical significance of the coefficient: *p<.10, **p<.05, and ***p<.01 
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